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Abstract 

A mathematical interpretation of the usual definition of entropy is given. This for- 
mulation makes some properties of entropy immediate. 

It seems illuminating to work simultaneously with two classical scenarios where entropy 
^ ' occurs: the set S of nonnegative real sequences with sum 1 (discrete probability distributions) 
and the set O of trace-class positive-definite (Hermitian) operators with trace 1 on a separable 
Hilbert space Ti (the quantum- mechanical counterpart of the former). In both cases the 
discussed set is a subset of a "|| ||i" Banach space (the sequence space l\ for S and the space 
of trace-class operators for O) with its norm || ||i, and is a subset of a "|| ||oo" Banach space 
(the sequence space ioo for S and the space B(TC) for O) with its norm || H^. 

In both cases the entropy of an u G S or O is defined as usual. For S 

O ! entropy (w) = - J2 w < ln (^i) > 

\0 . i 

and for O one applies the same formula for the sequence of eigenvalues. 

An "interpretation" for this expression may be given as follows: 

For u) G S orO, Define U(u) C R + x R + (where R + = [0,oo[ c R) as follows: a pair 
> '. ( r i, r oo), r i, r oo > 0, belongs to U(u) iff for two members of the |j ||i-Banach space with 
difference to, the open || ||i-ball of radius r\ centered in one of them and the open || | loo-ball 
5-i ' of radius r ro centered in the other do not intersect. 

Clearly, 

(r{,0 < (n.roc) G U(u) (r'^r'J G U{u). 

In other words, the boundary dU(uj) in R + x R + is the "graph" a of a non-increasing 
function F : r\ <-> (where in case of a jump the vertical segment spanning the jump is 
to be added to this "graph") and U(u) is the set of points in the first quadrant below this 
"graph" . 

In fact, it is easy to find U(u) explicitly: one easily checks for S, and, somewhat surpris- 
ingly, equally easily for O, that in both cases F is given by: 

(1) n = J2max(0, Ui - r^), 
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where in the O case the Ui are the eigenvalues. (In other words, a pair (ri,^) belongs to 
{/(a;) iff T\ is or less than the expression (1).) 

Thus, the function F is continuously decreasing from r\ = 0,^ = max(wj) = ||^||oo to 
^oo = 0, T\ = 1 and is piecewise linear, namely linear in intervals between the points where 
attains as values the uVs, the slope of w.r.t. r\ in such an interval being the reciprocal of 
the number n of the c<Vs greater than the range of in this interval. 

Now integrate ln(roo) dr\ over the whole dU(oo). An interval [r^r"] spoken about in the 
last paragraph, the values of at its endpoints being r"^ = < r' <yo = Ui (we ordered the 
Ui in non-increasing order), will contribute (note that here dr\ = n ■ dr^): 

n[u i+ i \n(u i+ i) - Ui ln(uji)] + r" - r[, 

and adding up all the intervals one get the sought-for expression for the entropy: 

(2) entropy (u) = — J ln(roo) dr\ — l. 

An immediate consequence of (2) is that any linear transformation of sequences (case S) 
or operators (case O), that both does not increase || ||oo and does not increase || ||i, is entropy 
non- decreasing (it enlarges balls, thus shrinks U(u), hence lowers dU(u>)). In particular, one 
deduces the fact, important in quantum theory, that for any fixed basis in the Hilbert space 
Ti, the operation of erasing all off-diagonal entries in the matrix of an oj e O is entropy non- 
decreasing. More generally, let X be a closed real linear subspace of the real (or Hermitian) 
elements of the || || i-Banach space (either for S or for O), so that Sx = X n S or Ox = X n O 
spans X. Suppose any oj e S'orO has a "conditional expectation" (necessarily unique) 

£ Sx or Ox, in the sense that 

Va G X (u x , a) = (w, a) , 

where in the S'-case (a, b) := J2 a ibi and in the O-case (a, b) := tr(afr). Then one has direct- 
sum decompositions of the real || ||i- and || Hoo-Banach spaces into the closures of X and of 
its annihilator, and the projection ux on X in these decompositions is norm-non-increasing 
for both norms. (Indeed, write each real/Hermitian a in the || ||i-space as a + uj + — a_u;_, 
a +) a_ > 0, a + + a_ = ||a||i, u> + ,u>- G S orO. Then ||7Tx(a)||i = ll^+^+x — ^-^-xlK — 
a + + a_ = 1 1 cz. 1 1 x . For another real/Hermitian b, (TT X (a),b) = (a,7Tx(b)) hence | (irx(a),b) \ < 
|| a lloo|| 7r x(&)||i < || a l|oo||^||i implying ||7rx(a)||oo < ll a lloo-) And one deduces that taking the 
above conditional expectation is entropy non-decreasing. 



